Marker layout method, mixed reality apparatus, and mixed reality space image generation method

ABSTRACT

There is disclosed a marker layout method in a mixed reality space, which can reliably detect markers in units of players, even when a plurality of players share a common mixed reality space. According to this invention, markers to be used by only a given player are laid out at positions that cannot be seen from other players. Real objects used in an application that uses a mixed reality space may be used.

FILED OF THE INVENTION

The present invention relates to a method of laying out a marker used asa position index upon presenting a mixed reality space and, moreparticularly, to a marker layout method which allows easy detection of amarker even when a plurality of players share a mixed reality space, anda mixed reality apparatus using the same.

BACKGROUND OF THE INVENTION

In recent years, Mixed Reality (MR) that aims at seamlessly coupling areal space and virtual space has been extensively studied. MR hasreceived a lot of attention as a technique which aims at realizingcoexistence of a virtual reality (VR) world that can be expressed inonly a situation isolated from a real space, and the real space, andaugments VR.

A typical apparatus for implementing MR is a head-mounted display (HMD).More specifically, MR is implemented by mixing and displaying real andvirtual spaces on the HMD. MR schemes using the HMD include an opticalsee-through scheme for superposing a CG image or the like onto an imagedisplayed on a semi-transparent (see-through) HMD, and a videosee-through scheme for mixing a CG image or the like onto image datasensed by a video camera attached to an HMD, and then displaying themixed image on the HMD.

MR can be used, quite possibly, in new fields which are qualitativelyquite different from VR, such as a medical assist application forpresenting the state inside the body of a patient to a doctor as if itwere seen through, a job assist application for superposing anddisplaying the assembly sequence of a product on real parts in afactory, and the like.

A technique for removing “deviation” between real and virtual spaces iscommonly required for these applications. “Deviation” can be classifiedinto positional deviation, temporal deviation, and qualitativedeviation, and many studies have been conventionally made for removal ofpositional deviation as the most fundamental requirement among thosedeviations.

Especially, in case of video see-through MR, since an image processscheme can be relatively easily applied as a method of correctingpositional deviation, alignment using an image process has beenconventionally proposed.

More specifically, a method in which markers which are marked in colorso an image process can easily detect are laid out at a predeterminedposition in a real space, and the viewpoint position is computed on thebasis of the marker positions detected from an image sensed by a cameraattached to a player, and a method of correcting an output signal from alocation/posture sensor based on marker positions in an image, and thelike are available.

When markers in image data are detected, and the location/posture of aplayer is estimated based on the detection result, the markers mustappear in an image to have an appropriate size and nearly uniformintervals. Also, since a sufficient number of markers must besimultaneously detected in the image upon computing the viewpointposition, the markers must be laid out to be observed in the image atsomewhat narrow intervals.

On the other hand, in order to improve the tracking or identificationprecision of markers, the markers must be laid out to be observed in theimage at somewhat broad intervals.

In case of a single player, it is not so difficult to lay out markers tosatisfy the aforementioned conditions. However, in an application whichallows a plurality of players to share a common MR space, markers whichare laid out at equal intervals at positions that can be observed from agiven player cannot often be observed at equal intervals from anotherplayer.

For this reason, Japanese Patent Laid-Open No. 11-84307 has proposed anarrangement in which in an air hockey game system in which two playershit and return a virtual puck toward each other on a table as a realobject, markers having different colors in units of players are providedto allow the individual players to observe the markers with a preferredlayout and size.

However, marker layout using different colors becomes harder withincreasing number of players who share a single MR space. Morespecifically, in order to detect a given color by an image process, thecolors of the markers and background object, and those of markers inunits of users must be easily detected and extracted by the imageprocess. However, when the number of colors used increases, it becomesdifficult to satisfy such conditions, and extraction errors andidentification errors among markers can occur.

FIG. 10 is a graph for explaining recognition error factors when aplurality of different color markers are used. In FIG. 10, the abscissaplots red, and the ordinate plots green. For the sake of simplicity, ablue axis is not shown. In FIG. 10, region A defines the colordistribution of marker type A (red marker), and region B defines thecolor distribution of marker type B (orange marker). In this manner,when the number of players increases and similar colors are used asmarkers, even though a player observes a red marker, the marker may bedetected as color of region B (i.e., of orange marker), thus causingdetection errors.

When a plurality of markers having different colors in units of playersare laid out, a very large number of markers appear in the real space,and may complicate the vision of the player, thus impairing reality uponexperiencing an MR space.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a marker layoutmethod in a mixed reality space, which can solve the aforementionedproblems, and can reliably detect markers in units of players even whena plurality of players share a common mixed reality space.

It is another object of the present invention to provide a marker layoutmethod in a mixed reality space, which can present the number of markersfrom simply increasing even when the number of players increases.

It is still another object of the present invention to provide a mixedreality apparatus that exploits the marker layout method in the mixedreality space according to the present invention.

It is still another object of the present invention to provide a mixedreality apparatus and a mixed reality space image generation method,which can solve the aforementioned problems, and allow a player toexperience mixed reality without being bothered by the presence ofmarkers.

More specifically, a gist of the present invention lies in a markerlayout method for laying out markers in a real space as position indicesupon generating a mixed reality space, comprising the step of: layingout the markers to have a positional relationship that allows a givenplayer not to observe markers to be used by only another player when aplurality of players who observe the mixed reality space withindifferent movable ranges observe the mixed reality space.

Another gist of the present invention lies in a mixed reality apparatusfor computing and/or correcting location/posture information of a playerusing markers laid out by a marker layout method of the presentinvention.

Still another gist of the present invention lies in a storage mediumthat stores a marker layout method in a mixed reality space according tothe present invention as a program which can be executed by a computer.

Still another gist of the present invention lies in a mixed realityapparatus for making a player experience mixed reality by making theplayer observe a mixed reality space image obtained by mixing real andvirtual spaces, markers serving as position indices being laid out inthe real space, the apparatus comprising: marker detection means fordetecting the markers from image data obtained by sensing the real spacefrom a substantially viewpoint position of the player; and mixed realityspace image generation means for generating the mixed reality spaceimage to be observed by the player, so the player observes virtualobject images that do not include any images of the markers insurrounding regions (marker regions) including the markers in the imagedata.

Still another gist of the present invention lies in a mixed realityspace image generation method for generating a mixed reality space imagewhich makes a player experience mixed reality by mixing a real space inwhich markers serving as position indices are laid out, and a virtualspace, comprising: the marker detection step of detecting the markersfrom image data obtained by sensing the real space from a substantiallyviewpoint position of the player; and the mixed reality space imagegeneration step of generating the mixed reality space image to beobserved by the player, so the player observes virtual object imagesthat do not include any images of the markers in surrounding regions(marker regions) including the markers in the image data.

Still another gist of the present invention lies in a storage mediumwhich stores a program that can be executed by a computer, and makes thecomputer that executes the program function as a mixed reality apparatusof the present invention.

Still another gist of the present invention lies in a storage mediumthat stores a mixed reality space image generation method according tothe present invention as a program which can be executed by a computer.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing an example of the arrangement of an MRapparatus to which a marker layout method according to the presentinvention can be applied;

FIG. 2 is a view for explaining the types and locations of devices theplayer wears;

FIGS. 3A and 3B are views for explaining a game in an embodiment of thepresent invention;

FIG. 4 is a view for explaining a marker layout method according to thepresent invention;

FIGS. 5A to 5C are views showing markers observed from the individualplayers in the layout shown in FIG. 4;

FIGS. 6A to 6C are views showing markers observed from the individualplayers when markers are laid out in the same layout as in FIG. 4without using any obstacles;

FIG. 7 is a graph showing the color region of a red marker in theembodiment of the present invention;

FIG. 8 is a flow chart showing the processing sequence for detecting amarker in the color region shown in FIG. 7 from color image data;

FIGS. 9A to 9C are views for explaining a marker deletion method; and

FIG. 10 is a graph for explaining detection errors when the number ofcolors of markers is increased.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A preferred embodiment of a marker layout method according to thepresent invention will be described hereinafter with reference to theaccompanying drawings. In this embodiment, the present invention isapplied to an MR game played by three players while sharing a singlevirtual space, but a marker layout method in an MR space according tothe present invention can be applied to any other arbitraryapplications.

FIG. 1 shows the arrangement of an MR apparatus for implementing an MRgame to which a marker layout method of the present invention isapplied. FIG. 1 shows the arrangement for three players. The MRapparatus comprises player processors 100 to 300 provided in units ofplayers, and a controller 400 connected to the player processors. Thenumber of player processors connected to the controller 400 increaseswith increasing number of players.

The player processors 100 to 300 have an identical arrangement. That is,each player processor comprises I/O blocks (102, 103, 105, 1011, 1021,1022) such as sensors, display device, and the like attached to theplayer's body, and circuit blocks (101, 104, 106, 107) for processingsignals captured by the sensors, and generating an image to be displayedon the display device on the basis of the signal processing results andinformation of other players supplied from the controller.

The arrangement of the player processor will be explained below withreference to FIG. 1 and FIG. 2 that shows the types and locations of I/Odevices each player wears in this embodiment.

As shown in FIG. 2, each player wears on his or her head a head-mountedimage display device (to be referred to as an HMD hereinafter) 105 as adisplay device for mixing and displaying real and virtual spaces. Inthis embodiment, the HMD can be either a video or optical see-throughtype. In the following description, a case using the video see-throughHMD will be explained.

Two small video cameras 103 are provided to those portions of the HMD105, that are near player's eyes. Video data sensed by these videocameras at nearly the same viewpoint as that of the player are suppliedto an image mixing unit 106 (to be described later) via an image inputunit 104, and are displayed on the HMD 105 of the player after beingsuperposed on a virtual space image. The image input unit 104 alsosupplies the input images to a head location detecting (measuring) unit101 in addition to the image mixing unit 106.

A head location/posture sensor 1011 comprising, e.g., a magnetic sensor,is also attached to the head of the player. The head location/posturesensor 1011 can be attached using, e.g., the HMD 105. The output signalfrom the head location/posture sensor 1011 is input to the headlocation/posture measuring unit 101. The head location/posture measuringunit 101 detects position correction markers using the image suppliedfrom the image input unit 104, and corrects the signal supplied from thehead location/posture sensor 1011, thus estimating the viewpointposition and posture of the player.

On the other hand, an interactive input device 102 is attached to aplayer's arm. The interactive input device 102 has a location/posturesensor 1021 for detecting the location and posture of a portion wherethe device 102 is attached, and a switch (trigger) 1022 which is turnedon/off in response to the motion of the attached portion and is used bythe player to input commands by making predetermined actions.

In the following description, a case will be exemplified below wherein aplurality of players (three players in this embodiment) share anidentical MR space, play a game for defeating enemies while ducking awayfrom attacks of enemies appearing in the MR space and competing withother players for the number of enemies he or she shoots down or scoreuntil the time or damage by enemy attacks reaches a given level, and caninput the following commands using the interactive input device 102.

Command 1 (Sight Command)

A command for displaying a line of sight indicating the sight positionin the virtual space. This command is input by locating the wrist abovethe elbow with the back of the hand facing up.

Command 2 (Shooting Command)

A command for shooting the sight position indicated by the line ofsight. This command is input by reciprocally moving the arm (from theelbow to the palm) back and forth at a given acceleration or higherwhile the line of sight is displayed by the sight command.

Command 3 (Defense Command)

A command for defending against an enemy's attack. This command is inputby directing the fingertips upward with the back of the hand facing theenemy.

Command 4 (Reset Command)

A command for re-inputting the sight command after the shooting commandis input. This command is input by having the arm hanging down.

More specifically, as routine actions in the game of this embodiment,the player repeats command inputs in a cycle of the sightcommand→shooting command→reset command→sight command, and inputs thedefense command as needed in this cycle.

The command input by the interactive input device 102 is supplied to animage generation unit 107.

The image generation unit 107 transfers the head location/postureinformation of the player supplied from the head location/posturemeasuring unit 101 and command information supplied from the interactiveinput device 102 to the controller 400. Also, the unit 107 generates avirtual space image to be displayed on the HMD 105 of the correspondingplayer using the head location/posture information and commandinformation of that player, and the head location/posture information,command information, and model information of other players, thelocation, moving direction, and state information of enemy characters,and information of the locations, shapes, and the like of obstacles laidout in the space, which are received from the controller 400, andoutputs the generated image to the image mixing unit 106.

The image mixing unit 106 mixes the image (real space image) captured bythe video cameras 103 attached at the positions near the view point ofthe player, and the virtual space image generated by the imagegeneration unit 107, and supplies the mixed image to the HMD 105 of theplayer.

The controller 400 comprises an MR space managing unit 1 to which theaforementioned player processors 100 to 300 are connected, and a realobject location measuring unit 2. The MR space managing unit 1distributes information that pertains to the head locations and posturesof the players, and the locations, postures, and commands of theinteractive input devices 102, which are received from the playerprocessors 100 to 300, and executes game processes such asappearance/disappearance control of enemy characters to be displayed inthe virtual space, scoring of shooting input by the shooting command,and the like. Information that pertains to the models, locations, movingdirections, and states (e.g., defeated or not) of enemy characters aredistributed to all the users connected together with each playerinformation.

When real objects 31 to 33 are laid out as obstacles to shooting toadjust the game difficulty level, as shown in FIGS. 3A and 3B, the MRspace managing unit 1 also manages information that pertains to theshapes and locations of these real objects 31 to 33.

FIG. 3A is a perspective view of a game field (MR space) viewed from acertain player, and FIG. 3B is a top view of the game field. FIGS. 3Aand 3B show a case wherein three real space objects 31 to 33 are laidout as obstacles on a table 30 (in the real space). In this embodiment,since the sight command is input by raising the wrist above the elbowusing the interactive input device 102, as described above, the realspace objects are laid out on the table 30 which is as high as the waistlevel of a standard body. However, the need for the table can beobviated depending on command input actions using the interactive inputdevice.

In FIGS. 3A and 3B, the real space objects 31 and 32 are fixed inposition, and the object 33 is movable. If a movable real space objectis provided, the game difficulty level can be dynamically changed. Forexample, when the movable real object 33 moves to the right and left ata random speed, it is more difficult for a player to shoot an enemycharacter as a target than a case wherein only the still real objects 31and 32 are laid out. The movement of the movable real object may becontrolled by either the MR space managing unit 1 or another controlcircuit. In this case, the model of the movable object is managed by theMR space managing unit 1, and its location is obtained by measuring anobject location sensor 331 provided to that movable object 33 by thereal object location measuring unit 2.

The aforementioned MR apparatus can be implemented by a client-serversystem which includes the controller 400 as a server, and (the circuitsections of) the player processors 100 to 300 as clients. Sinceprocesses that pertain to each player are distributed to and executed byeach client, the apparatus can flexibly cope with an increase/decreasein the number of players. More specifically, the player processor can beimplemented by a versatile computer having a video I/O function and asignal reception function from various sensors, and the controller 400can be implemented by a versatile computer having an interface that cancommunicate with each player processor, and a measurement signalreception function from the object location measuring unit 2.

However, since computations pertaining to three-dimensional imagedisplay must be done in real time, a relatively fast computer having anaccelerator dedicated to such computations (so-called 3D accelerator) orthe like is preferably used. Also, the controller 400 and playerprocessors 100 to 300 are preferably connected via communication lineshaving a large capacity such as 100BASE-T. If the communication line hasa small capacity, the processing speed drops larger with increasingnumber of players.

(Marker Layout Method)

FIG. 4 is a perspective view showing an example of a marker layout inthis embodiment. In the game of this embodiment, real objects serving asobstacles are laid out. By laying out markers using these obstacles, theaforementioned conditions can be satisfied while restricting markerswhich enter the field of view of each player within the moving range ofthe player.

FIGS. 5A to 5C show extracted markers players A, B, and C canrespectively see in FIG. 4. In this manner, since each player canobserve markers at nearly equal intervals and in a quantity required forcomputing the viewpoint position, but cannot see markers for otherplayers, the markers need not use different colors. Also, as shown inFIG. 4, a plurality of players can share identical markers.

FIGS. 6A to 6C show markers observed by the individual players incorrespondence with FIGS. 5A to 5C when no obstacles are used. As can beclearly understood from comparison between FIGS. 5A to 5C and FIGS. 6Ato 6C, the marker layout method according to the present invention canmake the number of markers each player observes very small, and satisfythe aforementioned conditions.

An increase in the number of players can be coped with by changing theshapes (sectional shapes, heights, and the like) of real objects towhich the markers are provided, or adding another color. Even when thenumber of colors increases, since one color is not assigned per player,markers for a large number of players can be laid out using fewercolors.

The marker layout positions may be determined manually but can bedetermined by generating real object models and viewpoint positionmovable range models of the individual players in advance, and obtaininga range in which a given player can see but the lines of sight of otherplayers are intercepted. Furthermore, the marker layout positions may becomputed using the number of markers to be provided and layout rules.Conversely, obstacle shapes and/or layout positions that satisfy theaforementioned conditions may be computed.

Even when no objects such as obstacles that can be used to lay outmarkers are available, real objects may be laid out at positions whereno problem is posed in terms of an application, and are hidden byvirtual space images, whereby the player can experience an MR spacewithout being bothered by the presence of real objects for markers.Hiding of markers and the like using virtual space images will beexplained in detail later.

(Detection of Marker)

A marker detection method will be explained below. FIG. 8 is a flowchart showing the flow of processing for detecting markers of type A(red markers) having color included in region A shown in FIG. 7. Thehead location/posture measuring unit 101 executes a marker detectionprocess and uses the detected marker information.

Images sensed by the video cameras 103 are captured via the image inputunit 104 (step S701). The captured images are then binarized (stepS702). More specifically, let 1 be pixels included in region A shown inFIG. 7 (a blue axis is not shown), and 0 be pixels included in theremaining region. That is, if

-   -   Ii: i-th pixel that forms input color image I    -   Ri, Gi, Bi: R, G, and B values that form Ii    -   ITHi: i-th pixel value of binary image    -   RminA, GminA, BminA: R, G, and B minimum values that define        region A    -   RmaxA, GmaxA, BmaxA: R, G, and B maximum values that define        region A        Then, a binary image ITH is formed by setting 1 in ITHi        corresponding to Ii that satisfies RminA<Ri<RmaxA,        GminA<Gi<GmaxA, and BminA<Bi<BmaxA, and 0 in ITHi corresponding        to another Ii in units of Ii.

The binary image ITH undergoes a labeling process to extract markerregions (clusters) (step S703). A center of gravity (Xn, Yn) and area anof each cluster are computed (step S704), and are output to a viewpointposition/posture estimation module (not shown) in the headlocation/posture measuring unit 101 (step S705). The viewpointposition/posture estimation module corrects the output signal from thehead location/posture sensor 1011 on the basis of absolute coordinatepositions of markers, which are registered in advance, and the markerpositions (centers of gravity of clusters) detected from the image, thuscomputing the viewpoint position and posture of the player.

In FIG. 8, a process upon detecting markers in a given color (red) hasbeen explained. When the number of players increases and there are aplurality of different color markers, detection processes in units ofcolors are repeated by changing a threshold value upon generating abinary image.

(Deletion of Marker)

As described above, markers are not necessary if the output from thehead location/posture sensor 1011 is accurate. Furthermore, the presenceof markers in an MR space is preferably not recognized since sense ofreality may suffer if the player recognizes markers.

For this reason, the present invention is characterized by apparentlydeleting markers to make the player unconscious of the presence ofmarkers. As a method of apparently deleting markers, various methods maybe used. Of these methods, a method of superposing a virtual image oneach marker and presenting the virtual image to the player is preferablesince it can minimize the processing load and unnatural feeling.

FIGS. 9A to 9C are views for explaining the marker deletion method. FIG.9A shows markers for player A, which are laid out according to themarker layout method in this embodiment, and corresponds to FIG. 5A. Inorder to delete markers laid out on real objects such as a table,obstacles, and the like, a prospective layout place is sensed using avideo camera, digital camera, still camera, or the like before markersare laid out so as to capture image data as a texture, and the markersare then laid out. The MR space managing unit 1 manages, as virtualobjects for hiding the markers, two-dimensional patches each of whichhas a position, direction, and size to hide each marker, and a textureimage corresponding to the marker before marker layout, and distributesthem to the player processors 100 to 300. The image generation unit 107in each play processor renders such patches as parts of a virtual spaceimage using the head location/posture information of the player uponrendering the virtual space image pertaining to the game. As a result,the image mixing unit 106 superposes images before marker layout onregions corresponding to the markers in the real space image (FIG. 9B).

In this manner, by replacing/superposing image data, the player canconcentrate on the game without being bothered by the presence of themarkers. Image data prepared as texture data are preferably capturedunder conditions (light source position and brightness) actually used toreduce unnatural feeling of vision. But texture images for deletingmarkers to be laid out on real objects made of an identical material mayuse a single common texture. Texture images are not particularly limitedas long as they do not make the player feel unnatural. Also, textureimages to be laid out for the game may be used to hide the markers, andthree-dimensional objects that hide the markers may be laid out asvirtual objects in place of two-dimensional patches.

In an application that does not use any obstacles, when real objects arelaid out for only the purpose of marker layout, or when it is hard tosuperpose virtual objects on individual markers, a virtual object 91that covers all real objects may be used, as shown in FIG. 9C. In suchcase, a virtual object to be used can be appropriately determineddepending on applications. For example, in a game application, a stageor something like that in a virtual space may be formed using thevirtual object 91. Or in an application in which the user is preferablyunaware of any difference from a real space, object data sensed in thereal space may be used.

Other Embodiments

In the above embodiment, information obtained from the markers is usedto correct errors of the head location/posture sensor 1011.Alternatively, the viewpoint position and posture of the player may becomputed from only the information obtained from the markers withoutusing the head location/posture sensor 1011, and the present inventioncan be used in such application.

The above embodiment has exemplified a video see-through MR apparatus,but the present invention can be similarly applied to an opticalsee-through MR apparatus.

In the above embodiment, single-color markers are used, but any othervisible features may be used. For example, a plurality of differentcolors may be used, or features other than color may be used. Also,specific textures, shapes, or the like may be used, and these featuresmay be combined.

The objects of the present invention are also achieved by supplying astorage medium (or recording medium), which records a program code of asoftware program that can implement the functions of the above-mentionedembodiments to a system or apparatus, and reading out and executing theprogram code stored in the storage medium by a computer (or a CPU orMPU) of the system or apparatus. In this case, the program code itselfread out from the storage medium implements the functions of theabove-mentioned embodiments, and the storage medium which stores theprogram code constitutes the present invention. The functions of theabove-mentioned embodiments may be implemented not only by executing thereadout program code by the computer but also by some or all of actualprocessing operations executed by an OS (operating system) running onthe computer on the basis of an instruction of the program code.

Furthermore, the functions of the above-mentioned embodiments may beimplemented by some or all of actual processing operations executed by aCPU or the like arranged in a function extension board or a functionextension unit, which is inserted in or connected to the computer, afterthe program code read out from the storage medium is written in a memoryof the extension board or unit.

To restate, according to the present invention, markers which are usedin an alignment process in an MR apparatus can be laid out whilesatisfying required positional and numerical conditions, even when aplurality of players share a single MR space.

Since the marker color need not be changed for each player, probabilityof occurrence of detection errors can be reduced even when the number ofplayers increases.

Furthermore, since real space objects which are laid out originally canbe used, no extra objects used to lay out markers are required dependingon applications.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

1-19. (canceled)
 20. An information processing method comprising: obtaining a model of a real object; obtaining information indicating a movable range of a viewpoint of a player; obtaining a layout condition of markers; and determining, based on the model of the real object, the information and the layout condition, positions at which the markers are to be laid out, wherein the markers are used for detecting the viewpoint and a posture of the player.
 21. The information processing method according to claim 20, wherein the layout condition is a layout rule.
 22. The information processing method according to claim 20, wherein the layout condition is the number of the markers.
 23. The information processing method according to claim 20, further comprising: storing, for each player, the information indicating a movable range of the viewpoint of a player.
 24. A computer-readable storage medium containing code for performing an information processing method comprising: obtaining a model of a real object; obtaining information indicating a movable range of a viewpoint of a player; obtaining a layout condition of markers; and determining, based on the model of the real object, the information and the layout condition, positions at which the markers are to be laid out, wherein the markers are used for detecting the viewpoint and a posture of the player.
 25. An information processing apparatus comprising: an obtaining unit, which obtains a model of a real object, information indicating a movable range of a viewpoint of a player, and a layout condition of markers; and a determining unit, which determines, based on the model of the real object, the information and the layout condition, positions at which the markers are to be laid out, wherein the markers are used for detecting the viewpoint and a posture of the player. 